Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce polymorphism resulting from unstable Node shapes #51682

Merged
merged 11 commits into from
Dec 13, 2022
Merged

Conversation

rbuckton
Copy link
Member

@rbuckton rbuckton commented Nov 29, 2022

We have a number of places in our codebase where we optionally attach properties to a Node. While a common practice in JS, this can sometimes have negative consequences with regards to runtime performance. Most modern JavaScript engines use Inline Caches (ICs) to optimize code paths, primarily when reading or writing to properties on objects, or to global variables.

Ideally, we want property ICs to be "monomorphic", such that they only ever see a single object shape (or "map" in V8 parlance) as "monomorphic" property ICs provide the fastest lookups. A "polymorphic" IC has slightly worse performance since it shifts from a one-to-one lookup to a short list of "map"-> property entries. If a new "map" is encountered once a polymorphic IC has around four entries, V8 will shift to a "megamporphic" IC. "Megamorphic" property ICs are the worst in terms of performance, as V8 essentially gives up on fast property access and relies on slow property lookups (there is still a cache, but it is cyclical: repeated property lookups for the same "map" will be fast but will age out as new "maps" are encountered).

Monomorphism isn't always possible, especially when accessing properties like .kind. However, in cases where we have branched on .kind, we should endeavor to ensure that further property accesses within that branch remain monomorphic, or at least polymorphic.

One way to achieve this is to ensure the shape of every Node is stable relative to its kind. This has the following implications:

  • Properties of Node and its subtypes really shouldn't be "optional" and should be fully initialized when a Node is produced (even if only with undefined).
  • Avoid casting nodes to things they aren't (there were several cases of node as any as JSDocContainer and node as TypeNode & { <optional properties> }).
  • Move properties off of supertypes that belong on subtypes (such as symbol, localSymbol, locals, nextContainer, flowNode, etc.), as that results in unchecked node.symbol accesses which often result in "wrong map" deoptimizations.
  • Avoid using "cover types" with a lot of optional properties in favor of union types where you must discriminate before access.
  • Avoid accessing properties other than kind when the node is a supertype or union, unless the function is only accessing one or two properties (like pos/end) since a megamorphic lookup on "pos" will have the same cost as a megamorpic lookup on "kind".
  • Where possible, a function or code path should try to target a single Node subtype, especially if it accesses multiple properties.

There is a trade-off for these changes, however. Pre-defining potentially unused properties on a Node increases the memory footprint. As such, if the memory cost becomes too severe, we may want to investigate removing properties from some Node subtypes where the value can be trivially recomputed from source or moved into a cache like NodeLinks or EmitNode.

@rbuckton
Copy link
Member Author

@typescript-bot perf test

@typescript-bot
Copy link
Collaborator

typescript-bot commented Nov 29, 2022

Heya @rbuckton, I've started to run the perf test suite on this PR at 70cdafc. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@rbuckton
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..51682
Metric main 51682 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 341,231k (± 0.02%) 364,522k (± 0.02%) +23,291k (+ 6.83%) 364,330k 364,631k
Parse Time 1.56s (± 0.92%) 1.56s (± 0.63%) -0.00s (- 0.06%) 1.54s 1.58s
Bind Time 0.53s (± 0.89%) 0.56s (± 1.20%) +0.02s (+ 4.50%) 0.54s 0.57s
Check Time 4.01s (± 1.11%) 3.95s (± 0.82%) -0.07s (- 1.64%) 3.89s 4.03s
Emit Time 4.29s (± 0.99%) 3.93s (± 1.47%) 🟩-0.36s (- 8.37%) 3.84s 4.13s
Total Time 10.39s (± 0.53%) 9.99s (± 0.77%) 🟩-0.40s (- 3.87%) 9.89s 10.27s
Compiler-Unions - node (v18.10.0, x64)
Memory used 187,342k (± 1.08%) 197,373k (± 0.65%) +10,031k (+ 5.35%) 196,734k 202,523k
Parse Time 0.62s (± 0.99%) 0.61s (± 1.19%) -0.01s (- 1.45%) 0.60s 0.63s
Bind Time 0.33s (± 1.34%) 0.35s (± 2.49%) +0.02s (+ 4.82%) 0.34s 0.37s
Check Time 5.05s (± 0.86%) 4.93s (± 0.78%) -0.12s (- 2.34%) 4.88s 5.04s
Emit Time 1.56s (± 0.63%) 1.46s (± 0.70%) 🟩-0.10s (- 6.59%) 1.44s 1.49s
Total Time 7.57s (± 0.54%) 7.35s (± 0.60%) -0.21s (- 2.84%) 7.28s 7.48s
Monaco - node (v18.10.0, x64)
Memory used 320,497k (± 0.01%) 347,249k (± 0.01%) +26,752k (+ 8.35%) 347,114k 347,334k
Parse Time 1.16s (± 1.79%) 1.18s (± 0.81%) +0.01s (+ 1.03%) 1.16s 1.19s
Bind Time 0.49s (± 1.40%) 0.50s (± 0.80%) +0.01s (+ 3.09%) 0.49s 0.51s
Check Time 3.85s (± 0.49%) 3.69s (± 0.25%) 🟩-0.16s (- 4.10%) 3.67s 3.71s
Emit Time 2.26s (± 1.00%) 2.04s (± 1.21%) 🟩-0.22s (- 9.62%) 2.00s 2.11s
Total Time 7.76s (± 0.50%) 7.41s (± 0.35%) 🟩-0.35s (- 4.49%) 7.34s 7.46s
TFS - node (v18.10.0, x64)
Memory used 283,711k (± 0.25%) 306,302k (± 0.23%) +22,591k (+ 7.96%) 304,838k 307,007k
Parse Time 0.97s (± 2.04%) 0.97s (± 1.55%) -0.00s (- 0.31%) 0.94s 1.00s
Bind Time 0.45s (± 6.89%) 0.52s (± 8.55%) +0.07s (+15.66%) 0.45s 0.59s
Check Time 3.79s (± 0.65%) 3.62s (± 0.59%) 🟩-0.17s (- 4.41%) 3.57s 3.67s
Emit Time 2.22s (± 0.88%) 2.02s (± 0.37%) 🟩-0.19s (- 8.75%) 2.01s 2.04s
Total Time 7.42s (± 0.82%) 7.13s (± 0.81%) 🟩-0.29s (- 3.96%) 6.99s 7.24s
material-ui - node (v18.10.0, x64)
Memory used 435,953k (± 0.02%) 451,491k (± 0.01%) +15,538k (+ 3.56%) 451,398k 451,623k
Parse Time 1.33s (± 0.74%) 1.32s (± 0.36%) -0.01s (- 0.68%) 1.31s 1.33s
Bind Time 0.49s (± 1.00%) 0.44s (± 1.02%) 🟩-0.06s (-11.34%) 0.43s 0.45s
Check Time 10.36s (± 0.90%) 10.22s (± 0.80%) -0.15s (- 1.42%) 10.10s 10.46s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 12.19s (± 0.85%) 11.98s (± 0.68%) -0.21s (- 1.74%) 11.85s 12.22s
xstate - node (v18.10.0, x64)
Memory used 518,397k (± 0.01%) 553,113k (± 0.01%) +34,716k (+ 6.70%) 552,966k 553,340k
Parse Time 1.91s (± 0.61%) 1.92s (± 0.52%) +0.01s (+ 0.63%) 1.90s 1.94s
Bind Time 0.77s (± 3.38%) 0.75s (± 2.99%) -0.02s (- 2.46%) 0.69s 0.79s
Check Time 1.04s (± 0.73%) 1.02s (± 0.85%) -0.02s (- 1.74%) 1.00s 1.04s
Emit Time 0.05s (± 0.00%) 0.05s (± 0.00%) 0.00s ( 0.00%) 0.05s 0.05s
Total Time 3.77s (± 1.00%) 3.74s (± 0.71%) -0.03s (- 0.85%) 3.66s 3.79s
Angular - node (v16.17.1, x64)
Memory used 340,608k (± 0.02%) 363,911k (± 0.02%) +23,303k (+ 6.84%) 363,727k 364,005k
Parse Time 1.90s (± 0.36%) 1.91s (± 0.58%) +0.02s (+ 1.06%) 1.90s 1.95s
Bind Time 0.65s (± 0.72%) 0.67s (± 0.77%) +0.02s (+ 3.23%) 0.66s 0.68s
Check Time 5.18s (± 0.39%) 5.02s (± 0.80%) 🟩-0.16s (- 3.05%) 4.93s 5.10s
Emit Time 5.14s (± 0.67%) 4.71s (± 1.81%) 🟩-0.43s (- 8.32%) 4.57s 5.01s
Total Time 12.87s (± 0.39%) 12.32s (± 0.77%) 🟩-0.54s (- 4.23%) 12.10s 12.53s
Compiler-Unions - node (v16.17.1, x64)
Memory used 187,354k (± 0.53%) 200,511k (± 0.60%) +13,156k (+ 7.02%) 198,477k 201,977k
Parse Time 0.79s (± 1.03%) 0.79s (± 0.38%) -0.00s (- 0.13%) 0.79s 0.80s
Bind Time 0.42s (± 1.12%) 0.44s (± 0.91%) +0.02s (+ 3.78%) 0.43s 0.45s
Check Time 6.09s (± 0.48%) 5.89s (± 1.05%) 🟩-0.20s (- 3.25%) 5.80s 6.11s
Emit Time 1.93s (± 0.49%) 1.80s (± 1.21%) 🟩-0.12s (- 6.33%) 1.78s 1.88s
Total Time 9.23s (± 0.40%) 8.93s (± 0.73%) 🟩-0.31s (- 3.32%) 8.80s 9.14s
Monaco - node (v16.17.1, x64)
Memory used 319,816k (± 0.01%) 346,546k (± 0.02%) +26,730k (+ 8.36%) 346,386k 346,654k
Parse Time 1.43s (± 0.47%) 1.43s (± 0.47%) -0.01s (- 0.42%) 1.41s 1.44s
Bind Time 0.60s (± 0.61%) 0.61s (± 0.85%) +0.02s (+ 2.52%) 0.60s 0.62s
Check Time 4.89s (± 0.35%) 4.69s (± 0.48%) 🟩-0.20s (- 3.99%) 4.65s 4.74s
Emit Time 2.74s (± 0.74%) 2.49s (± 0.81%) 🟩-0.26s (- 9.30%) 2.42s 2.53s
Total Time 9.66s (± 0.32%) 9.22s (± 0.47%) 🟩-0.43s (- 4.50%) 9.13s 9.34s
TFS - node (v16.17.1, x64)
Memory used 282,283k (± 0.01%) 304,161k (± 0.01%) +21,879k (+ 7.75%) 304,117k 304,192k
Parse Time 1.17s (± 0.71%) 1.16s (± 0.57%) -0.00s (- 0.26%) 1.15s 1.18s
Bind Time 0.66s (± 3.82%) 0.59s (± 4.04%) 🟩-0.07s (-10.41%) 0.58s 0.69s
Check Time 4.77s (± 0.38%) 4.59s (± 0.41%) 🟩-0.18s (- 3.84%) 4.53s 4.62s
Emit Time 2.79s (± 1.88%) 2.52s (± 1.61%) 🟩-0.27s (- 9.58%) 2.47s 2.62s
Total Time 9.39s (± 0.78%) 8.87s (± 0.53%) 🟩-0.52s (- 5.55%) 8.81s 9.00s
material-ui - node (v16.17.1, x64)
Memory used 435,264k (± 0.00%) 450,960k (± 0.01%) +15,696k (+ 3.61%) 450,867k 451,070k
Parse Time 1.65s (± 0.75%) 1.67s (± 0.75%) +0.01s (+ 0.85%) 1.64s 1.69s
Bind Time 0.50s (± 1.15%) 0.51s (± 1.14%) +0.00s (+ 0.80%) 0.50s 0.52s
Check Time 11.87s (± 0.66%) 11.78s (± 0.78%) -0.09s (- 0.76%) 11.61s 11.93s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.02s (± 0.61%) 13.95s (± 0.67%) -0.07s (- 0.49%) 13.77s 14.13s
xstate - node (v16.17.1, x64)
Memory used 515,991k (± 0.01%) 550,680k (± 0.01%) +34,690k (+ 6.72%) 550,577k 550,859k
Parse Time 2.32s (± 0.59%) 2.33s (± 0.81%) +0.02s (+ 0.65%) 2.28s 2.37s
Bind Time 0.84s (± 1.28%) 0.84s (± 2.65%) -0.00s (- 0.36%) 0.80s 0.90s
Check Time 1.36s (± 0.77%) 1.33s (± 0.65%) -0.03s (- 1.91%) 1.31s 1.35s
Emit Time 0.06s (± 0.00%) 0.06s (± 0.00%) 0.00s ( 0.00%) 0.06s 0.06s
Total Time 4.58s (± 0.54%) 4.57s (± 0.51%) -0.01s (- 0.24%) 4.51s 4.64s
Angular - node (v14.15.1, x64)
Memory used 334,102k (± 0.01%) 357,633k (± 0.01%) +23,531k (+ 7.04%) 357,587k 357,708k
Parse Time 2.07s (± 0.75%) 2.04s (± 0.64%) -0.03s (- 1.35%) 2.02s 2.08s
Bind Time 0.70s (± 0.84%) 0.72s (± 0.56%) +0.01s (+ 2.13%) 0.71s 0.73s
Check Time 5.54s (± 0.71%) 5.35s (± 0.31%) 🟩-0.20s (- 3.57%) 5.31s 5.38s
Emit Time 5.38s (± 0.72%) 4.80s (± 0.80%) 🟩-0.57s (-10.67%) 4.73s 4.88s
Total Time 13.70s (± 0.50%) 12.91s (± 0.40%) 🟩-0.79s (- 5.73%) 12.79s 13.05s
Compiler-Unions - node (v14.15.1, x64)
Memory used 182,679k (± 0.63%) 196,207k (± 0.52%) +13,528k (+ 7.41%) 193,420k 196,950k
Parse Time 0.89s (± 0.82%) 0.90s (± 0.66%) +0.01s (+ 0.90%) 0.88s 0.91s
Bind Time 0.46s (± 1.09%) 0.47s (± 0.85%) +0.02s (+ 3.52%) 0.46s 0.48s
Check Time 6.36s (± 0.55%) 6.10s (± 0.53%) 🟩-0.26s (- 4.05%) 6.03s 6.17s
Emit Time 2.07s (± 1.36%) 1.91s (± 0.49%) 🟩-0.16s (- 7.87%) 1.88s 1.93s
Total Time 9.77s (± 0.50%) 9.38s (± 0.33%) 🟩-0.40s (- 4.08%) 9.31s 9.45s
Monaco - node (v14.15.1, x64)
Memory used 314,635k (± 0.01%) 341,509k (± 0.00%) +26,874k (+ 8.54%) 341,472k 341,544k
Parse Time 1.58s (± 0.60%) 1.58s (± 0.67%) -0.00s (- 0.06%) 1.56s 1.61s
Bind Time 0.64s (± 0.58%) 0.64s (± 0.56%) +0.01s (+ 1.42%) 0.64s 0.65s
Check Time 5.19s (± 0.49%) 4.98s (± 0.56%) 🟩-0.21s (- 4.06%) 4.94s 5.04s
Emit Time 2.91s (± 0.83%) 2.57s (± 0.87%) 🟩-0.34s (-11.62%) 2.53s 2.62s
Total Time 10.32s (± 0.37%) 9.78s (± 0.48%) 🟩-0.54s (- 5.25%) 9.71s 9.89s
TFS - node (v14.15.1, x64)
Memory used 279,369k (± 0.01%) 301,278k (± 0.00%) +21,909k (+ 7.84%) 301,251k 301,312k
Parse Time 1.33s (± 1.19%) 1.33s (± 1.49%) +0.00s (+ 0.30%) 1.31s 1.40s
Bind Time 0.59s (± 0.68%) 0.61s (± 1.96%) +0.02s (+ 4.07%) 0.60s 0.66s
Check Time 5.07s (± 0.65%) 4.86s (± 0.34%) 🟩-0.21s (- 4.09%) 4.83s 4.91s
Emit Time 3.08s (± 0.78%) 2.78s (± 1.27%) 🟩-0.30s (- 9.61%) 2.73s 2.87s
Total Time 10.07s (± 0.59%) 9.59s (± 0.44%) 🟩-0.48s (- 4.78%) 9.49s 9.67s
material-ui - node (v14.15.1, x64)
Memory used 430,750k (± 0.00%) 446,409k (± 0.01%) +15,658k (+ 3.64%) 446,328k 446,454k
Parse Time 1.87s (± 0.67%) 1.87s (± 0.36%) +0.00s (+ 0.11%) 1.86s 1.89s
Bind Time 0.53s (± 1.40%) 0.55s (± 0.85%) +0.02s (+ 3.19%) 0.54s 0.56s
Check Time 12.29s (± 0.73%) 12.17s (± 0.75%) -0.13s (- 1.04%) 11.98s 12.32s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.70s (± 0.63%) 14.59s (± 0.64%) -0.11s (- 0.75%) 14.40s 14.77s
xstate - node (v14.15.1, x64)
Memory used 504,228k (± 0.00%) 539,071k (± 0.00%) +34,844k (+ 6.91%) 539,033k 539,141k
Parse Time 2.63s (± 0.50%) 2.65s (± 0.64%) +0.02s (+ 0.65%) 2.62s 2.70s
Bind Time 0.84s (± 0.62%) 0.83s (± 0.60%) -0.01s (- 0.71%) 0.82s 0.84s
Check Time 1.48s (± 0.61%) 1.47s (± 0.61%) -0.01s (- 0.61%) 1.44s 1.48s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.02s (± 0.34%) 5.02s (± 0.45%) -0.00s (- 0.04%) 4.96s 5.08s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51682 10
Baseline main 10

TSServer

Comparison Report - main..51682
Metric main 51682 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,057ms (± 1.02%) 1,058ms (± 0.87%) +1ms (+ 0.09%) 1,040ms 1,079ms
Req 2 - geterr 2,592ms (± 1.16%) 2,533ms (± 0.92%) -59ms (- 2.29%) 2,482ms 2,574ms
Req 3 - references 166ms (± 0.57%) 164ms (± 0.73%) -1ms (- 0.78%) 161ms 167ms
Req 4 - navto 140ms (± 1.75%) 143ms (± 1.70%) +4ms (+ 2.51%) 139ms 151ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 62ms (± 2.20%) 53ms (± 7.31%) 🟩-8ms (-13.59%) 48ms 62ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,106ms (± 0.79%) 1,138ms (± 0.47%) +32ms (+ 2.87%) 1,129ms 1,156ms
Req 2 - geterr 1,596ms (± 0.64%) 1,571ms (± 0.62%) -25ms (- 1.57%) 1,553ms 1,591ms
Req 3 - references 170ms (± 0.84%) 172ms (± 0.91%) +3ms (+ 1.59%) 170ms 177ms
Req 4 - navto 151ms (± 0.59%) 153ms (± 1.04%) +2ms (+ 1.06%) 150ms 156ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 54ms (± 1.89%) 54ms (± 1.41%) +0ms (+ 0.19%) 52ms 56ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,513ms (± 0.62%) 1,566ms (± 1.10%) +53ms (+ 3.51%) 1,536ms 1,620ms
Req 2 - geterr 555ms (± 1.14%) 535ms (± 0.83%) 🟩-21ms (- 3.69%) 523ms 540ms
Req 3 - references 58ms (± 1.16%) 57ms (± 2.44%) -1ms (- 2.08%) 54ms 60ms
Req 4 - navto 197ms (± 0.66%) 193ms (± 0.66%) -3ms (- 1.73%) 191ms 195ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 212ms (± 0.80%) 215ms (± 1.65%) +2ms (+ 1.04%) 205ms 222ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,310ms (± 0.53%) 1,329ms (± 0.55%) +19ms (+ 1.41%) 1,312ms 1,346ms
Req 2 - geterr 3,195ms (± 0.71%) 3,161ms (± 0.99%) -34ms (- 1.06%) 3,077ms 3,244ms
Req 3 - references 192ms (± 0.83%) 195ms (± 1.57%) +3ms (+ 1.77%) 189ms 204ms
Req 4 - navto 152ms (± 0.98%) 155ms (± 0.97%) +3ms (+ 1.64%) 151ms 157ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 63ms (± 5.11%) 58ms (± 1.53%) 🟩-5ms (- 7.30%) 57ms 60ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,393ms (± 0.74%) 1,421ms (± 0.67%) +28ms (+ 2.00%) 1,399ms 1,438ms
Req 2 - geterr 2,109ms (± 0.48%) 2,058ms (± 0.59%) -51ms (- 2.42%) 2,033ms 2,079ms
Req 3 - references 200ms (± 0.45%) 202ms (± 0.64%) +2ms (+ 0.95%) 198ms 203ms
Req 4 - navto 167ms (± 1.45%) 168ms (± 1.17%) +1ms (+ 0.60%) 165ms 173ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 56ms (± 0.71%) 57ms (± 1.32%) +1ms (+ 1.07%) 55ms 58ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,834ms (± 0.36%) 1,877ms (± 0.72%) +43ms (+ 2.33%) 1,840ms 1,904ms
Req 2 - geterr 725ms (± 0.49%) 699ms (± 0.30%) 🟩-26ms (- 3.61%) 694ms 704ms
Req 3 - references 68ms (± 0.77%) 68ms (± 1.77%) -0ms (- 0.29%) 65ms 70ms
Req 4 - navto 200ms (± 0.58%) 196ms (± 0.95%) -4ms (- 1.80%) 191ms 200ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 254ms (± 0.69%) 258ms (± 1.28%) +4ms (+ 1.49%) 252ms 266ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,461ms (± 0.52%) 1,471ms (± 0.82%) +10ms (+ 0.65%) 1,451ms 1,501ms
Req 2 - geterr 3,447ms (± 0.78%) 3,355ms (± 0.54%) -92ms (- 2.66%) 3,314ms 3,397ms
Req 3 - references 206ms (± 0.18%) 210ms (± 0.85%) +4ms (+ 1.74%) 207ms 214ms
Req 4 - navto 163ms (± 0.38%) 170ms (± 0.98%) +7ms (+ 4.17%) 166ms 174ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 58ms (± 1.29%) 59ms (± 1.61%) +1ms (+ 1.73%) 57ms 62ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,537ms (± 0.45%) 1,572ms (± 0.24%) +35ms (+ 2.29%) 1,562ms 1,580ms
Req 2 - geterr 2,287ms (± 0.47%) 2,205ms (± 0.36%) 🟩-82ms (- 3.57%) 2,187ms 2,227ms
Req 3 - references 217ms (± 0.84%) 219ms (± 0.92%) +2ms (+ 0.97%) 215ms 224ms
Req 4 - navto 175ms (± 0.99%) 179ms (± 0.72%) +4ms (+ 2.52%) 176ms 181ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 56ms (± 1.47%) 60ms (± 2.45%) +4ms (+ 6.21%) 57ms 63ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,018ms (± 0.86%) 1,987ms (± 0.49%) -31ms (- 1.53%) 1,973ms 2,019ms
Req 2 - geterr 750ms (± 0.63%) 737ms (± 0.65%) -13ms (- 1.76%) 730ms 750ms
Req 3 - references 73ms (± 1.23%) 73ms (± 1.15%) +1ms (+ 0.69%) 72ms 76ms
Req 4 - navto 219ms (± 0.64%) 212ms (± 0.94%) 🟩-7ms (- 3.33%) 208ms 218ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 273ms (± 1.66%) 281ms (± 1.46%) +7ms (+ 2.71%) 272ms 288ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51682 10
Baseline main 10

Startup

Comparison Report - main..51682
Metric main 51682 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 120.48ms (± 0.61%) 117.75ms (± 0.43%) -2.73ms (- 2.26%) 115.38ms 129.08ms
tsserver-startup - node (v16.17.1, x64)
Execution time 201.08ms (± 0.48%) 199.17ms (± 0.40%) -1.91ms (- 0.95%) 195.18ms 210.37ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 195.97ms (± 0.39%) 193.30ms (± 0.43%) -2.67ms (- 1.36%) 189.50ms 201.28ms
typescript-startup - node (v16.17.1, x64)
Execution time 179.65ms (± 0.45%) 179.68ms (± 0.44%) +0.02ms (+ 0.01%) 174.98ms 185.97ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51682 10
Baseline main 10

Developer Information:

Download Benchmark

@@ -84,3 +84,4 @@ tests/cases/user/puppeteer/puppeteer
tests/cases/user/axios-src/axios-src
tests/cases/user/prettier/prettier
.eslintcache
*v8.log
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a question, if I may, what kind of tools you were using to investigate those things? I imagine that in the case of Nodes the issue was already known for some time but I wonder how I could test those things in my libraries.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have an internal tool I wrote to analyze log files generated by various V8 commandline options.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Andarist One open source tool that might be of interest is https://github.com/thlorenz/deoptigate

Copy link
Member

@sandersn sandersn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me*, just a few detail questions.

*Looking at the functionality changes -- I'll take your word that the performance is better.

src/compiler/parser.ts Outdated Show resolved Hide resolved
src/compiler/utilities.ts Show resolved Hide resolved
src/compiler/utilities.ts Show resolved Hide resolved
src/compiler/factory/nodeFactory.ts Outdated Show resolved Hide resolved
src/compiler/factory/nodeFactory.ts Outdated Show resolved Hide resolved
src/compiler/factory/nodeFactory.ts Outdated Show resolved Hide resolved
src/compiler/binder.ts Outdated Show resolved Hide resolved
src/compiler/checker.ts Outdated Show resolved Hide resolved
src/compiler/checker.ts Outdated Show resolved Hide resolved
src/services/refactors/convertExport.ts Show resolved Hide resolved
@rbuckton
Copy link
Member Author

rbuckton commented Dec 6, 2022

@typescript-bot perf test

Running again after rebasing against main in case something changed.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 6, 2022

Heya @rbuckton, I've started to run the perf test suite on this PR at f476065. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@rbuckton
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..51682
Metric main 51682 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 341,094k (± 0.01%) 364,492k (± 0.02%) +23,398k (+ 6.86%) 364,274k 364,558k
Parse Time 1.55s (± 0.91%) 1.56s (± 0.58%) +0.01s (+ 0.32%) 1.54s 1.58s
Bind Time 0.52s (± 1.06%) 0.54s (± 0.88%) +0.02s (+ 4.02%) 0.53s 0.55s
Check Time 3.99s (± 0.51%) 3.90s (± 0.51%) -0.09s (- 2.25%) 3.87s 3.95s
Emit Time 4.26s (± 1.07%) 3.90s (± 0.46%) 🟩-0.36s (- 8.44%) 3.87s 3.96s
Total Time 10.34s (± 0.48%) 9.91s (± 0.23%) 🟩-0.43s (- 4.14%) 9.85s 9.95s
Compiler-Unions - node (v18.10.0, x64)
Memory used 187,849k (± 1.10%) 199,082k (± 1.06%) +11,233k (+ 5.98%) 196,633k 202,606k
Parse Time 0.61s (± 1.23%) 0.61s (± 0.95%) -0.00s (- 0.16%) 0.60s 0.62s
Bind Time 0.33s (± 1.14%) 0.35s (± 1.49%) +0.02s (+ 7.38%) 0.34s 0.36s
Check Time 5.02s (± 0.53%) 4.93s (± 0.70%) -0.09s (- 1.87%) 4.86s 5.02s
Emit Time 1.55s (± 0.91%) 1.46s (± 1.27%) 🟩-0.10s (- 6.24%) 1.42s 1.51s
Total Time 7.52s (± 0.35%) 7.35s (± 0.64%) -0.17s (- 2.25%) 7.26s 7.45s
Monaco - node (v18.10.0, x64)
Memory used 320,437k (± 0.02%) 347,167k (± 0.02%) +26,730k (+ 8.34%) 347,003k 347,306k
Parse Time 1.14s (± 0.96%) 1.17s (± 0.77%) +0.03s (+ 2.37%) 1.15s 1.18s
Bind Time 0.48s (± 1.09%) 0.51s (± 1.74%) +0.03s (+ 5.85%) 0.49s 0.53s
Check Time 3.84s (± 0.50%) 3.70s (± 0.47%) 🟩-0.14s (- 3.68%) 3.66s 3.75s
Emit Time 2.24s (± 0.69%) 2.05s (± 0.88%) 🟩-0.19s (- 8.31%) 2.00s 2.08s
Total Time 7.69s (± 0.23%) 7.42s (± 0.35%) 🟩-0.27s (- 3.56%) 7.33s 7.46s
TFS - node (v18.10.0, x64)
Memory used 283,508k (± 0.24%) 305,468k (± 0.23%) +21,960k (+ 7.75%) 304,697k 306,941k
Parse Time 0.95s (± 0.50%) 0.94s (± 1.17%) -0.01s (- 0.74%) 0.93s 0.98s
Bind Time 0.47s (± 8.80%) 0.56s (± 5.11%) +0.09s (+20.30%) 0.45s 0.59s
Check Time 3.79s (± 0.99%) 3.62s (± 0.32%) 🟩-0.17s (- 4.38%) 3.60s 3.65s
Emit Time 2.20s (± 1.04%) 2.01s (± 0.68%) 🟩-0.19s (- 8.46%) 1.99s 2.05s
Total Time 7.40s (± 0.63%) 7.14s (± 0.31%) 🟩-0.26s (- 3.57%) 7.08s 7.17s
material-ui - node (v18.10.0, x64)
Memory used 435,974k (± 0.01%) 451,646k (± 0.01%) +15,672k (+ 3.59%) 451,548k 451,853k
Parse Time 1.31s (± 0.76%) 1.34s (± 1.15%) +0.03s (+ 2.13%) 1.32s 1.37s
Bind Time 0.49s (± 0.68%) 0.42s (± 1.77%) 🟩-0.07s (-14.29%) 0.41s 0.44s
Check Time 10.26s (± 0.87%) 10.14s (± 0.45%) -0.12s (- 1.13%) 10.08s 10.27s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 12.06s (± 0.74%) 11.90s (± 0.34%) -0.16s (- 1.30%) 11.86s 12.02s
xstate - node (v18.10.0, x64)
Memory used 518,650k (± 0.01%) 553,440k (± 0.01%) +34,791k (+ 6.71%) 553,358k 553,593k
Parse Time 1.88s (± 0.64%) 1.88s (± 0.36%) +0.01s (+ 0.37%) 1.87s 1.90s
Bind Time 0.70s (± 2.08%) 0.72s (± 2.79%) +0.02s (+ 2.73%) 0.67s 0.76s
Check Time 1.04s (± 0.55%) 1.02s (± 1.00%) -0.02s (- 1.83%) 1.00s 1.05s
Emit Time 0.05s (± 0.00%) 0.05s (± 0.00%) 0.00s ( 0.00%) 0.05s 0.05s
Total Time 3.67s (± 0.53%) 3.68s (± 0.53%) +0.01s (+ 0.30%) 3.65s 3.73s
Angular - node (v16.17.1, x64)
Memory used 340,507k (± 0.02%) 363,820k (± 0.01%) +23,312k (+ 6.85%) 363,672k 363,860k
Parse Time 1.86s (± 0.38%) 1.90s (± 0.72%) +0.04s (+ 2.31%) 1.88s 1.94s
Bind Time 0.65s (± 0.56%) 0.67s (± 1.14%) +0.02s (+ 3.10%) 0.66s 0.69s
Check Time 5.14s (± 0.59%) 5.02s (± 0.47%) -0.12s (- 2.32%) 4.97s 5.08s
Emit Time 5.08s (± 0.60%) 4.69s (± 0.82%) 🟩-0.39s (- 7.66%) 4.61s 4.77s
Total Time 12.72s (± 0.41%) 12.28s (± 0.52%) 🟩-0.45s (- 3.53%) 12.15s 12.40s
Compiler-Unions - node (v16.17.1, x64)
Memory used 188,325k (± 0.65%) 199,862k (± 0.61%) +11,537k (+ 6.13%) 198,494k 202,021k
Parse Time 0.78s (± 0.74%) 0.79s (± 0.84%) +0.01s (+ 0.89%) 0.78s 0.80s
Bind Time 0.42s (± 0.82%) 0.44s (± 1.09%) +0.02s (+ 4.80%) 0.43s 0.45s
Check Time 6.05s (± 0.48%) 5.91s (± 0.82%) -0.14s (- 2.28%) 5.82s 6.02s
Emit Time 1.93s (± 0.75%) 1.80s (± 0.54%) 🟩-0.13s (- 6.69%) 1.78s 1.82s
Total Time 9.18s (± 0.40%) 8.94s (± 0.69%) -0.24s (- 2.65%) 8.83s 9.09s
Monaco - node (v16.17.1, x64)
Memory used 319,791k (± 0.01%) 346,535k (± 0.01%) +26,744k (+ 8.36%) 346,460k 346,579k
Parse Time 1.40s (± 0.53%) 1.42s (± 0.58%) +0.02s (+ 1.43%) 1.40s 1.44s
Bind Time 0.59s (± 0.75%) 0.61s (± 0.78%) +0.02s (+ 2.53%) 0.60s 0.62s
Check Time 4.84s (± 0.43%) 4.66s (± 0.65%) 🟩-0.18s (- 3.64%) 4.60s 4.74s
Emit Time 2.71s (± 0.65%) 2.47s (± 0.84%) 🟩-0.24s (- 8.75%) 2.43s 2.51s
Total Time 9.54s (± 0.36%) 9.16s (± 0.37%) 🟩-0.38s (- 3.97%) 9.07s 9.23s
TFS - node (v16.17.1, x64)
Memory used 282,300k (± 0.01%) 304,182k (± 0.01%) +21,882k (+ 7.75%) 304,138k 304,212k
Parse Time 1.15s (± 0.83%) 1.18s (± 1.12%) +0.04s (+ 3.23%) 1.15s 1.21s
Bind Time 0.65s (± 4.83%) 0.58s (± 1.29%) 🟩-0.07s (-10.68%) 0.56s 0.59s
Check Time 4.77s (± 0.44%) 4.58s (± 0.29%) 🟩-0.19s (- 4.08%) 4.56s 4.61s
Emit Time 2.79s (± 2.02%) 2.52s (± 2.02%) 🟩-0.27s (- 9.57%) 2.43s 2.62s
Total Time 9.35s (± 0.74%) 8.86s (± 0.59%) 🟩-0.49s (- 5.28%) 8.77s 8.95s
material-ui - node (v16.17.1, x64)
Memory used 435,348k (± 0.01%) 451,036k (± 0.02%) +15,688k (+ 3.60%) 450,909k 451,232k
Parse Time 1.62s (± 0.42%) 1.64s (± 0.34%) +0.03s (+ 1.61%) 1.63s 1.65s
Bind Time 0.50s (± 0.72%) 0.49s (± 0.68%) -0.01s (- 2.78%) 0.48s 0.50s
Check Time 11.79s (± 1.16%) 11.64s (± 0.66%) -0.15s (- 1.25%) 11.49s 11.85s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 13.91s (± 0.98%) 13.77s (± 0.55%) -0.14s (- 0.97%) 13.61s 13.97s
xstate - node (v16.17.1, x64)
Memory used 516,237k (± 0.01%) 550,972k (± 0.01%) +34,736k (+ 6.73%) 550,838k 551,161k
Parse Time 2.26s (± 0.59%) 2.27s (± 0.63%) +0.01s (+ 0.40%) 2.23s 2.29s
Bind Time 0.82s (± 1.18%) 0.81s (± 1.56%) -0.01s (- 1.22%) 0.78s 0.84s
Check Time 1.34s (± 0.54%) 1.33s (± 0.71%) -0.01s (- 0.60%) 1.32s 1.35s
Emit Time 0.06s (± 0.00%) 0.06s (± 0.00%) 0.00s ( 0.00%) 0.06s 0.06s
Total Time 4.48s (± 0.22%) 4.47s (± 0.38%) -0.01s (- 0.18%) 4.43s 4.50s
Angular - node (v14.15.1, x64)
Memory used 333,996k (± 0.01%) 357,677k (± 0.00%) +23,680k (+ 7.09%) 357,649k 357,719k
Parse Time 2.04s (± 0.64%) 2.03s (± 0.66%) -0.00s (- 0.10%) 2.01s 2.07s
Bind Time 0.70s (± 0.52%) 0.71s (± 0.67%) +0.01s (+ 1.58%) 0.70s 0.72s
Check Time 5.47s (± 0.43%) 5.34s (± 0.22%) -0.13s (- 2.36%) 5.32s 5.37s
Emit Time 5.32s (± 0.72%) 4.78s (± 0.42%) 🟩-0.54s (-10.21%) 4.74s 4.84s
Total Time 13.52s (± 0.36%) 12.86s (± 0.26%) 🟩-0.66s (- 4.87%) 12.82s 12.97s
Compiler-Unions - node (v14.15.1, x64)
Memory used 183,047k (± 0.68%) 196,508k (± 0.38%) +13,461k (+ 7.35%) 193,530k 196,873k
Parse Time 0.89s (± 0.67%) 0.90s (± 0.75%) +0.01s (+ 1.24%) 0.88s 0.91s
Bind Time 0.46s (± 1.32%) 0.47s (± 0.78%) +0.01s (+ 2.42%) 0.46s 0.47s
Check Time 6.36s (± 0.45%) 6.12s (± 0.27%) 🟩-0.24s (- 3.74%) 6.08s 6.16s
Emit Time 2.06s (± 0.82%) 1.89s (± 0.71%) 🟩-0.17s (- 8.39%) 1.86s 1.92s
Total Time 9.76s (± 0.32%) 9.37s (± 0.30%) 🟩-0.39s (- 4.02%) 9.30s 9.45s
Monaco - node (v14.15.1, x64)
Memory used 314,576k (± 0.01%) 341,514k (± 0.01%) +26,938k (+ 8.56%) 341,477k 341,550k
Parse Time 1.56s (± 0.72%) 1.57s (± 0.45%) +0.01s (+ 0.90%) 1.56s 1.59s
Bind Time 0.63s (± 0.75%) 0.65s (± 0.77%) +0.02s (+ 2.38%) 0.64s 0.66s
Check Time 5.20s (± 0.44%) 4.98s (± 0.52%) 🟩-0.22s (- 4.29%) 4.92s 5.04s
Emit Time 2.88s (± 0.34%) 2.59s (± 1.12%) 🟩-0.29s (-10.20%) 2.54s 2.67s
Total Time 10.26s (± 0.29%) 9.78s (± 0.50%) 🟩-0.49s (- 4.76%) 9.69s 9.94s
TFS - node (v14.15.1, x64)
Memory used 279,383k (± 0.01%) 301,308k (± 0.01%) +21,925k (+ 7.85%) 301,250k 301,352k
Parse Time 1.31s (± 0.55%) 1.31s (± 0.51%) -0.00s (- 0.38%) 1.30s 1.33s
Bind Time 0.59s (± 0.62%) 0.60s (± 0.56%) +0.02s (+ 2.90%) 0.60s 0.61s
Check Time 5.08s (± 0.60%) 4.88s (± 0.29%) 🟩-0.21s (- 4.05%) 4.85s 4.90s
Emit Time 3.08s (± 0.59%) 2.78s (± 0.59%) 🟩-0.30s (- 9.62%) 2.75s 2.82s
Total Time 10.06s (± 0.43%) 9.57s (± 0.17%) 🟩-0.49s (- 4.90%) 9.53s 9.60s
material-ui - node (v14.15.1, x64)
Memory used 430,799k (± 0.01%) 446,659k (± 0.03%) +15,860k (+ 3.68%) 446,345k 446,793k
Parse Time 1.85s (± 0.43%) 1.87s (± 0.62%) +0.01s (+ 0.76%) 1.85s 1.90s
Bind Time 0.53s (± 0.42%) 0.54s (± 0.89%) +0.01s (+ 1.13%) 0.53s 0.55s
Check Time 12.24s (± 0.53%) 12.02s (± 0.63%) -0.22s (- 1.81%) 11.89s 12.23s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.62s (± 0.46%) 14.42s (± 0.55%) -0.20s (- 1.40%) 14.28s 14.63s
xstate - node (v14.15.1, x64)
Memory used 504,618k (± 0.01%) 539,565k (± 0.01%) +34,947k (+ 6.93%) 539,519k 539,660k
Parse Time 2.54s (± 0.40%) 2.58s (± 0.68%) +0.04s (+ 1.42%) 2.55s 2.61s
Bind Time 0.83s (± 0.44%) 0.81s (± 0.64%) 🟩-0.03s (- 3.11%) 0.80s 0.82s
Check Time 1.47s (± 0.35%) 1.45s (± 0.48%) -0.02s (- 1.23%) 1.44s 1.47s
Emit Time 0.07s (± 0.00%) 0.07s (± 3.23%) -0.00s (- 1.43%) 0.06s 0.07s
Total Time 4.92s (± 0.26%) 4.92s (± 0.39%) -0.00s (- 0.08%) 4.87s 4.96s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51682 10
Baseline main 10

TSServer

Comparison Report - main..51682
Metric main 51682 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,053ms (± 0.88%) 1,057ms (± 0.67%) +4ms (+ 0.39%) 1,040ms 1,077ms
Req 2 - geterr 2,587ms (± 0.60%) 2,545ms (± 0.78%) -42ms (- 1.62%) 2,502ms 2,587ms
Req 3 - references 165ms (± 0.62%) 165ms (± 0.55%) -0ms (- 0.12%) 163ms 167ms
Req 4 - navto 138ms (± 1.13%) 142ms (± 0.78%) +4ms (+ 2.60%) 140ms 145ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 60ms (± 4.33%) 55ms (± 8.59%) 🟩-5ms (- 7.82%) 49ms 62ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,100ms (± 0.45%) 1,132ms (± 0.38%) +32ms (+ 2.91%) 1,120ms 1,140ms
Req 2 - geterr 1,599ms (± 0.59%) 1,566ms (± 0.63%) -33ms (- 2.07%) 1,545ms 1,598ms
Req 3 - references 168ms (± 0.54%) 174ms (± 1.10%) +7ms (+ 3.87%) 169ms 177ms
Req 4 - navto 151ms (± 0.83%) 152ms (± 0.88%) +1ms (+ 0.60%) 149ms 155ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 53ms (± 1.13%) 52ms (± 1.30%) -0ms (- 0.38%) 51ms 54ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,508ms (± 0.52%) 1,534ms (± 1.08%) +26ms (+ 1.72%) 1,492ms 1,564ms
Req 2 - geterr 554ms (± 0.70%) 544ms (± 1.29%) -10ms (- 1.75%) 527ms 557ms
Req 3 - references 59ms (± 2.35%) 60ms (± 2.16%) +1ms (+ 1.88%) 56ms 62ms
Req 4 - navto 195ms (± 0.54%) 193ms (± 0.60%) -2ms (- 1.23%) 191ms 195ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 215ms (± 1.76%) 212ms (± 1.07%) -3ms (- 1.21%) 207ms 216ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,296ms (± 0.46%) 1,316ms (± 0.35%) +20ms (+ 1.57%) 1,302ms 1,326ms
Req 2 - geterr 3,187ms (± 0.71%) 3,154ms (± 0.90%) -33ms (- 1.02%) 3,086ms 3,223ms
Req 3 - references 191ms (± 0.90%) 194ms (± 1.05%) +3ms (+ 1.62%) 191ms 199ms
Req 4 - navto 151ms (± 0.63%) 155ms (± 0.95%) +4ms (+ 2.72%) 152ms 159ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 64ms (±13.05%) 58ms (± 2.06%) 🟩-6ms (- 9.55%) 56ms 62ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,384ms (± 0.61%) 1,411ms (± 0.55%) +26ms (+ 1.89%) 1,391ms 1,426ms
Req 2 - geterr 2,105ms (± 0.31%) 2,071ms (± 0.68%) -33ms (- 1.58%) 2,041ms 2,098ms
Req 3 - references 199ms (± 0.82%) 201ms (± 0.47%) +2ms (+ 1.01%) 199ms 203ms
Req 4 - navto 166ms (± 1.67%) 165ms (± 0.48%) -1ms (- 0.30%) 163ms 167ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 56ms (± 2.89%) 56ms (± 1.47%) +0ms (+ 0.18%) 55ms 59ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,811ms (± 0.38%) 1,858ms (± 0.45%) +47ms (+ 2.58%) 1,839ms 1,875ms
Req 2 - geterr 724ms (± 0.31%) 689ms (± 0.50%) 🟩-34ms (- 4.74%) 681ms 695ms
Req 3 - references 68ms (± 1.48%) 66ms (± 0.98%) -2ms (- 2.22%) 65ms 68ms
Req 4 - navto 199ms (± 0.73%) 196ms (± 1.18%) -3ms (- 1.46%) 191ms 201ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 254ms (± 0.99%) 253ms (± 1.18%) -1ms (- 0.35%) 249ms 262ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,451ms (± 0.41%) 1,478ms (± 0.29%) +26ms (+ 1.81%) 1,470ms 1,487ms
Req 2 - geterr 3,435ms (± 0.66%) 3,357ms (± 0.43%) -77ms (- 2.25%) 3,326ms 3,392ms
Req 3 - references 206ms (± 0.65%) 208ms (± 0.80%) +3ms (+ 1.36%) 205ms 212ms
Req 4 - navto 162ms (± 0.73%) 169ms (± 1.10%) +6ms (+ 3.95%) 164ms 173ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 57ms (± 1.04%) 58ms (± 1.37%) +1ms (+ 1.39%) 57ms 61ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,525ms (± 0.43%) 1,570ms (± 0.43%) +45ms (+ 2.98%) 1,559ms 1,589ms
Req 2 - geterr 2,288ms (± 0.41%) 2,209ms (± 0.44%) 🟩-79ms (- 3.43%) 2,194ms 2,245ms
Req 3 - references 218ms (± 1.02%) 221ms (± 0.89%) +4ms (+ 1.75%) 218ms 227ms
Req 4 - navto 173ms (± 0.85%) 179ms (± 0.50%) +5ms (+ 3.12%) 177ms 181ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 56ms (± 0.93%) 60ms (± 5.70%) +4ms (+ 6.80%) 56ms 72ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,993ms (± 0.50%) 1,967ms (± 0.37%) -26ms (- 1.31%) 1,954ms 1,981ms
Req 2 - geterr 745ms (± 0.42%) 738ms (± 0.50%) -7ms (- 0.97%) 730ms 746ms
Req 3 - references 72ms (± 0.86%) 74ms (± 1.38%) +2ms (+ 2.50%) 72ms 76ms
Req 4 - navto 218ms (± 0.45%) 212ms (± 0.41%) -6ms (- 2.66%) 211ms 214ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 270ms (± 1.16%) 279ms (± 1.48%) +9ms (+ 3.41%) 271ms 288ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51682 10
Baseline main 10

Startup

Comparison Report - main..51682
Metric main 51682 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 117.61ms (± 0.39%) 117.24ms (± 0.38%) -0.37ms (- 0.32%) 115.28ms 124.36ms
tsserver-startup - node (v16.17.1, x64)
Execution time 197.48ms (± 0.29%) 197.03ms (± 0.30%) -0.45ms (- 0.23%) 194.18ms 212.27ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 191.93ms (± 0.30%) 191.74ms (± 0.31%) -0.19ms (- 0.10%) 188.43ms 200.56ms
typescript-startup - node (v16.17.1, x64)
Execution time 177.12ms (± 0.27%) 177.56ms (± 0.39%) +0.44ms (+ 0.25%) 174.81ms 184.54ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51682 10
Baseline main 10

Developer Information:

Download Benchmark

@rbuckton
Copy link
Member Author

rbuckton commented Dec 7, 2022

I'm going to temporarily borrow this PR to see if the benefit I saw locally in #51788 was a knock-on effect from the changes in this PR, since #51788 isn't having the same impact in benchmarks as I was expecting. If it does improve, I'll leave it in and close the other one.

@rbuckton
Copy link
Member Author

rbuckton commented Dec 7, 2022

@typescript-bot perf test

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 7, 2022

Heya @rbuckton, I've started to run the perf test suite on this PR at 5f84fc6. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@rbuckton
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..51682
Metric main 51682 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 341,113k (± 0.01%) 364,455k (± 0.02%) +23,342k (+ 6.84%) 364,285k 364,556k
Parse Time 1.57s (± 0.79%) 1.58s (± 0.89%) +0.01s (+ 0.83%) 1.55s 1.61s
Bind Time 0.52s (± 0.69%) 0.55s (± 1.17%) +0.03s (+ 5.34%) 0.54s 0.57s
Check Time 4.03s (± 0.63%) 3.91s (± 0.92%) -0.11s (- 2.85%) 3.83s 3.99s
Emit Time 4.28s (± 0.99%) 3.97s (± 1.34%) 🟩-0.30s (- 7.09%) 3.85s 4.07s
Total Time 10.40s (± 0.61%) 10.03s (± 0.85%) 🟩-0.37s (- 3.58%) 9.85s 10.18s
Compiler-Unions - node (v18.10.0, x64)
Memory used 187,917k (± 1.11%) 199,605k (± 1.08%) +11,688k (+ 6.22%) 196,425k 202,519k
Parse Time 0.61s (± 0.95%) 0.63s (± 1.32%) +0.01s (+ 2.12%) 0.61s 0.65s
Bind Time 0.33s (± 1.53%) 0.35s (± 0.84%) +0.03s (+ 8.31%) 0.35s 0.36s
Check Time 4.99s (± 0.61%) 4.97s (± 0.67%) -0.02s (- 0.38%) 4.87s 5.04s
Emit Time 1.55s (± 0.68%) 1.46s (± 1.35%) 🟩-0.09s (- 5.73%) 1.43s 1.50s
Total Time 7.48s (± 0.55%) 7.42s (± 0.65%) -0.07s (- 0.90%) 7.27s 7.51s
Monaco - node (v18.10.0, x64)
Memory used 320,542k (± 0.05%) 347,224k (± 0.01%) +26,682k (+ 8.32%) 347,118k 347,307k
Parse Time 1.16s (± 1.76%) 1.18s (± 0.95%) +0.03s (+ 2.25%) 1.16s 1.21s
Bind Time 0.48s (± 1.20%) 0.51s (± 1.46%) +0.03s (+ 5.59%) 0.50s 0.53s
Check Time 3.84s (± 0.65%) 3.67s (± 0.55%) 🟩-0.17s (- 4.43%) 3.63s 3.71s
Emit Time 2.25s (± 0.79%) 2.06s (± 0.98%) 🟩-0.19s (- 8.45%) 2.01s 2.09s
Total Time 7.72s (± 0.40%) 7.42s (± 0.61%) 🟩-0.30s (- 3.94%) 7.32s 7.50s
TFS - node (v18.10.0, x64)
Memory used 283,509k (± 0.23%) 305,436k (± 0.23%) +21,928k (+ 7.73%) 304,687k 306,931k
Parse Time 0.96s (± 0.92%) 0.96s (± 1.09%) +0.00s (+ 0.10%) 0.94s 0.99s
Bind Time 0.45s (± 7.33%) 0.57s (± 2.60%) +0.12s (+26.49%) 0.53s 0.59s
Check Time 3.80s (± 0.50%) 3.40s (± 0.53%) 🟩-0.40s (-10.53%) 3.34s 3.42s
Emit Time 2.21s (± 0.85%) 2.04s (± 1.12%) 🟩-0.16s (- 7.43%) 2.00s 2.09s
Total Time 7.42s (± 0.48%) 6.97s (± 0.67%) 🟩-0.45s (- 6.01%) 6.85s 7.06s
material-ui - node (v18.10.0, x64)
Memory used 436,020k (± 0.02%) 451,578k (± 0.01%) +15,559k (+ 3.57%) 451,516k 451,641k
Parse Time 1.32s (± 0.77%) 1.35s (± 0.99%) +0.04s (+ 2.96%) 1.33s 1.39s
Bind Time 0.49s (± 1.64%) 0.43s (± 2.56%) 🟩-0.05s (-11.07%) 0.41s 0.45s
Check Time 10.32s (± 0.65%) 10.27s (± 0.92%) -0.05s (- 0.50%) 10.07s 10.44s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 12.13s (± 0.59%) 12.06s (± 0.78%) -0.07s (- 0.57%) 11.87s 12.22s
xstate - node (v18.10.0, x64)
Memory used 518,646k (± 0.01%) 553,384k (± 0.01%) +34,738k (+ 6.70%) 553,291k 553,578k
Parse Time 1.88s (± 0.72%) 1.92s (± 0.65%) +0.04s (+ 2.23%) 1.89s 1.95s
Bind Time 0.69s (± 1.80%) 0.72s (± 1.48%) +0.02s (+ 3.17%) 0.69s 0.74s
Check Time 1.04s (± 0.83%) 1.03s (± 1.12%) -0.00s (- 0.48%) 1.01s 1.05s
Emit Time 0.05s (± 0.00%) 0.05s (± 0.00%) 0.00s ( 0.00%) 0.05s 0.05s
Total Time 3.67s (± 0.52%) 3.73s (± 0.69%) +0.06s (+ 1.58%) 3.67s 3.78s
Angular - node (v16.17.1, x64)
Memory used 340,529k (± 0.01%) 363,781k (± 0.02%) +23,253k (+ 6.83%) 363,571k 363,887k
Parse Time 1.86s (± 0.64%) 1.93s (± 0.83%) +0.06s (+ 3.49%) 1.89s 1.96s
Bind Time 0.64s (± 1.06%) 0.67s (± 0.51%) +0.02s (+ 3.57%) 0.66s 0.67s
Check Time 5.15s (± 0.70%) 5.04s (± 0.28%) -0.11s (- 2.12%) 5.01s 5.08s
Emit Time 5.13s (± 1.44%) 4.71s (± 1.33%) 🟩-0.42s (- 8.19%) 4.57s 4.81s
Total Time 12.78s (± 0.80%) 12.34s (± 0.73%) 🟩-0.44s (- 3.46%) 12.19s 12.51s
Compiler-Unions - node (v16.17.1, x64)
Memory used 187,680k (± 0.59%) 200,514k (± 0.62%) +12,834k (+ 6.84%) 198,430k 202,000k
Parse Time 0.80s (± 1.27%) 0.80s (± 0.70%) +0.00s (+ 0.38%) 0.79s 0.81s
Bind Time 0.42s (± 1.76%) 0.44s (± 1.11%) +0.02s (+ 4.96%) 0.43s 0.45s
Check Time 6.13s (± 1.58%) 5.93s (± 0.67%) 🟩-0.20s (- 3.29%) 5.84s 6.05s
Emit Time 1.95s (± 1.56%) 1.81s (± 0.94%) 🟩-0.15s (- 7.43%) 1.78s 1.84s
Total Time 9.30s (± 1.44%) 8.98s (± 0.49%) 🟩-0.32s (- 3.47%) 8.86s 9.08s
Monaco - node (v16.17.1, x64)
Memory used 319,822k (± 0.05%) 346,520k (± 0.01%) +26,698k (+ 8.35%) 346,361k 346,589k
Parse Time 1.41s (± 0.95%) 1.45s (± 0.79%) +0.04s (+ 2.91%) 1.42s 1.47s
Bind Time 0.59s (± 0.76%) 0.61s (± 0.80%) +0.03s (+ 4.42%) 0.61s 0.63s
Check Time 4.90s (± 1.15%) 4.62s (± 0.38%) 🟩-0.27s (- 5.60%) 4.59s 4.66s
Emit Time 2.72s (± 0.59%) 2.50s (± 1.11%) 🟩-0.22s (- 8.20%) 2.44s 2.59s
Total Time 9.61s (± 0.60%) 9.18s (± 0.41%) 🟩-0.44s (- 4.54%) 9.08s 9.29s
TFS - node (v16.17.1, x64)
Memory used 282,296k (± 0.01%) 304,116k (± 0.02%) +21,820k (+ 7.73%) 303,950k 304,218k
Parse Time 1.15s (± 0.84%) 1.19s (± 1.15%) +0.04s (+ 3.21%) 1.17s 1.24s
Bind Time 0.63s (± 5.73%) 0.58s (± 1.06%) 🟩-0.05s (- 8.07%) 0.57s 0.59s
Check Time 4.77s (± 0.36%) 4.33s (± 0.38%) 🟩-0.44s (- 9.20%) 4.29s 4.36s
Emit Time 2.74s (± 1.85%) 2.49s (± 0.70%) 🟩-0.25s (- 9.12%) 2.46s 2.53s
Total Time 9.29s (± 0.76%) 8.59s (± 0.35%) 🟩-0.70s (- 7.58%) 8.52s 8.64s
material-ui - node (v16.17.1, x64)
Memory used 435,337k (± 0.01%) 450,978k (± 0.01%) +15,641k (+ 3.59%) 450,919k 451,122k
Parse Time 1.62s (± 0.55%) 1.68s (± 0.42%) +0.06s (+ 3.96%) 1.67s 1.70s
Bind Time 0.51s (± 0.94%) 0.50s (± 1.04%) -0.01s (- 1.18%) 0.49s 0.51s
Check Time 11.80s (± 0.91%) 11.75s (± 0.59%) -0.04s (- 0.36%) 11.59s 11.89s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 13.92s (± 0.79%) 13.94s (± 0.54%) +0.02s (+ 0.11%) 13.77s 14.08s
xstate - node (v16.17.1, x64)
Memory used 516,206k (± 0.01%) 550,964k (± 0.01%) +34,758k (+ 6.73%) 550,864k 551,120k
Parse Time 2.27s (± 0.42%) 2.33s (± 0.60%) +0.06s (+ 2.82%) 2.30s 2.36s
Bind Time 0.82s (± 1.15%) 0.82s (± 2.29%) -0.00s (- 0.24%) 0.77s 0.86s
Check Time 1.35s (± 0.79%) 1.34s (± 0.35%) -0.01s (- 0.59%) 1.33s 1.35s
Emit Time 0.06s (± 0.00%) 0.06s (± 0.00%) 0.00s ( 0.00%) 0.06s 0.06s
Total Time 4.50s (± 0.51%) 4.56s (± 0.44%) +0.06s (+ 1.27%) 4.51s 4.59s
Angular - node (v14.15.1, x64)
Memory used 334,006k (± 0.00%) 357,653k (± 0.01%) +23,647k (+ 7.08%) 357,599k 357,694k
Parse Time 2.04s (± 0.54%) 2.06s (± 0.39%) +0.02s (+ 1.08%) 2.05s 2.08s
Bind Time 0.70s (± 1.27%) 0.71s (± 0.51%) +0.02s (+ 2.44%) 0.71s 0.72s
Check Time 5.50s (± 0.31%) 5.37s (± 0.46%) -0.13s (- 2.44%) 5.30s 5.41s
Emit Time 5.38s (± 0.47%) 4.83s (± 0.60%) 🟩-0.56s (-10.33%) 4.76s 4.91s
Total Time 13.62s (± 0.31%) 12.97s (± 0.37%) 🟩-0.65s (- 4.78%) 12.83s 13.11s
Compiler-Unions - node (v14.15.1, x64)
Memory used 183,635k (± 0.65%) 196,534k (± 0.39%) +12,900k (+ 7.02%) 193,417k 196,969k
Parse Time 0.89s (± 0.58%) 0.91s (± 1.04%) +0.02s (+ 2.02%) 0.89s 0.93s
Bind Time 0.46s (± 0.82%) 0.47s (± 0.77%) +0.02s (+ 4.18%) 0.47s 0.48s
Check Time 6.35s (± 0.61%) 6.18s (± 0.42%) -0.17s (- 2.74%) 6.14s 6.26s
Emit Time 2.05s (± 0.96%) 1.91s (± 0.57%) 🟩-0.14s (- 6.78%) 1.89s 1.93s
Total Time 9.75s (± 0.56%) 9.47s (± 0.37%) -0.28s (- 2.86%) 9.42s 9.57s
Monaco - node (v14.15.1, x64)
Memory used 314,612k (± 0.01%) 341,514k (± 0.01%) +26,902k (+ 8.55%) 341,475k 341,551k
Parse Time 1.56s (± 0.37%) 1.58s (± 0.59%) +0.02s (+ 1.28%) 1.57s 1.60s
Bind Time 0.63s (± 0.47%) 0.65s (± 1.15%) +0.02s (+ 2.37%) 0.64s 0.67s
Check Time 5.17s (± 0.25%) 4.94s (± 0.48%) 🟩-0.23s (- 4.41%) 4.90s 4.99s
Emit Time 2.88s (± 0.63%) 2.60s (± 0.66%) 🟩-0.28s (- 9.68%) 2.57s 2.64s
Total Time 10.24s (± 0.27%) 9.77s (± 0.33%) 🟩-0.47s (- 4.59%) 9.71s 9.83s
TFS - node (v14.15.1, x64)
Memory used 279,374k (± 0.01%) 301,284k (± 0.01%) +21,910k (+ 7.84%) 301,253k 301,329k
Parse Time 1.32s (± 1.13%) 1.33s (± 1.08%) +0.00s (+ 0.23%) 1.30s 1.36s
Bind Time 0.59s (± 0.84%) 0.61s (± 1.06%) +0.02s (+ 3.75%) 0.60s 0.62s
Check Time 5.10s (± 0.44%) 4.62s (± 0.35%) 🟩-0.47s (- 9.30%) 4.58s 4.65s
Emit Time 3.09s (± 0.72%) 2.77s (± 0.79%) 🟩-0.32s (-10.39%) 2.73s 2.82s
Total Time 10.10s (± 0.44%) 9.33s (± 0.45%) 🟩-0.77s (- 7.62%) 9.22s 9.43s
material-ui - node (v14.15.1, x64)
Memory used 430,824k (± 0.01%) 446,772k (± 0.00%) +15,948k (+ 3.70%) 446,748k 446,810k
Parse Time 1.86s (± 0.51%) 1.89s (± 0.64%) +0.03s (+ 1.83%) 1.87s 1.93s
Bind Time 0.53s (± 0.69%) 0.54s (± 0.63%) +0.02s (+ 3.23%) 0.54s 0.55s
Check Time 12.27s (± 0.60%) 12.15s (± 0.73%) -0.12s (- 0.98%) 11.98s 12.42s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.66s (± 0.54%) 14.58s (± 0.63%) -0.07s (- 0.51%) 14.39s 14.85s
xstate - node (v14.15.1, x64)
Memory used 504,513k (± 0.01%) 539,559k (± 0.01%) +35,046k (+ 6.95%) 539,421k 539,739k
Parse Time 2.58s (± 0.70%) 2.63s (± 0.67%) +0.05s (+ 1.82%) 2.60s 2.68s
Bind Time 0.84s (± 0.44%) 0.82s (± 0.57%) -0.02s (- 1.91%) 0.81s 0.83s
Check Time 1.47s (± 0.60%) 1.45s (± 0.74%) -0.02s (- 1.43%) 1.43s 1.48s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 4.96s (± 0.31%) 4.97s (± 0.48%) +0.01s (+ 0.28%) 4.92s 5.03s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51682 10
Baseline main 10

TSServer

Comparison Report - main..51682
Metric main 51682 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,054ms (± 0.83%) 1,069ms (± 0.59%) +15ms (+ 1.39%) 1,056ms 1,081ms
Req 2 - geterr 2,596ms (± 0.79%) 2,564ms (± 1.03%) -32ms (- 1.23%) 2,523ms 2,645ms
Req 3 - references 165ms (± 0.39%) 165ms (± 0.63%) -0ms (- 0.24%) 162ms 167ms
Req 4 - navto 138ms (± 0.64%) 143ms (± 1.04%) +5ms (+ 3.63%) 140ms 146ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 61ms (± 2.00%) 57ms (± 7.93%) 🟩-4ms (- 6.57%) 49ms 63ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,108ms (± 0.80%) 1,138ms (± 0.49%) +30ms (+ 2.73%) 1,124ms 1,147ms
Req 2 - geterr 1,605ms (± 0.56%) 1,570ms (± 0.81%) -35ms (- 2.18%) 1,544ms 1,607ms
Req 3 - references 168ms (± 0.55%) 173ms (± 1.30%) +5ms (+ 3.03%) 170ms 180ms
Req 4 - navto 152ms (± 0.58%) 154ms (± 0.66%) +2ms (+ 1.25%) 152ms 156ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 53ms (± 1.13%) 53ms (± 1.32%) +1ms (+ 0.95%) 51ms 54ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 1,507ms (± 0.55%) 1,553ms (± 0.99%) +45ms (+ 3.01%) 1,524ms 1,576ms
Req 2 - geterr 553ms (± 0.55%) 545ms (± 1.25%) -9ms (- 1.59%) 535ms 561ms
Req 3 - references 58ms (± 1.04%) 58ms (± 2.46%) +0ms (+ 0.70%) 54ms 61ms
Req 4 - navto 196ms (± 0.93%) 196ms (± 1.14%) -0ms (- 0.20%) 191ms 202ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 213ms (± 1.68%) 213ms (± 1.10%) +1ms (+ 0.24%) 209ms 218ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,300ms (± 0.54%) 1,335ms (± 0.53%) +35ms (+ 2.72%) 1,321ms 1,348ms
Req 2 - geterr 3,209ms (± 0.80%) 3,174ms (± 0.91%) -35ms (- 1.10%) 3,120ms 3,254ms
Req 3 - references 192ms (± 0.78%) 195ms (± 0.73%) +4ms (+ 1.88%) 192ms 198ms
Req 4 - navto 151ms (± 0.71%) 157ms (± 0.53%) +5ms (+ 3.57%) 154ms 158ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 61ms (± 3.95%) 58ms (± 1.62%) 🟩-3ms (- 5.38%) 57ms 60ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,386ms (± 1.03%) 1,423ms (± 0.64%) +37ms (+ 2.69%) 1,403ms 1,445ms
Req 2 - geterr 2,113ms (± 0.52%) 2,076ms (± 0.36%) -37ms (- 1.75%) 2,055ms 2,087ms
Req 3 - references 198ms (± 0.58%) 202ms (± 0.66%) +4ms (+ 2.17%) 200ms 206ms
Req 4 - navto 164ms (± 0.76%) 167ms (± 0.73%) +3ms (+ 1.58%) 164ms 170ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 57ms (± 2.65%) 57ms (± 1.14%) +0ms (+ 0.53%) 55ms 58ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 1,823ms (± 0.45%) 1,880ms (± 0.40%) +57ms (+ 3.10%) 1,864ms 1,895ms
Req 2 - geterr 725ms (± 0.74%) 696ms (± 0.64%) 🟩-29ms (- 4.04%) 683ms 704ms
Req 3 - references 68ms (± 0.98%) 66ms (± 1.21%) 🟩-2ms (- 3.07%) 65ms 68ms
Req 4 - navto 201ms (± 0.68%) 197ms (± 1.01%) -4ms (- 1.74%) 193ms 201ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 254ms (± 0.99%) 255ms (± 1.18%) +2ms (+ 0.67%) 248ms 261ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,451ms (± 0.64%) 1,481ms (± 0.62%) +30ms (+ 2.06%) 1,464ms 1,506ms
Req 2 - geterr 3,435ms (± 0.75%) 3,377ms (± 0.64%) -58ms (- 1.70%) 3,332ms 3,417ms
Req 3 - references 205ms (± 0.83%) 209ms (± 0.46%) +3ms (+ 1.51%) 207ms 211ms
Req 4 - navto 162ms (± 0.91%) 170ms (± 0.55%) +8ms (+ 4.81%) 167ms 171ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 57ms (± 1.08%) 60ms (± 5.17%) +3ms (+ 5.62%) 57ms 72ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,524ms (± 0.26%) 1,582ms (± 0.65%) +59ms (+ 3.84%) 1,554ms 1,607ms
Req 2 - geterr 2,302ms (± 0.56%) 2,214ms (± 0.39%) 🟩-87ms (- 3.79%) 2,203ms 2,243ms
Req 3 - references 218ms (± 1.04%) 222ms (± 1.12%) +5ms (+ 2.16%) 219ms 230ms
Req 4 - navto 175ms (± 0.46%) 179ms (± 0.97%) +4ms (+ 2.23%) 175ms 183ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 56ms (± 0.90%) 61ms (± 5.93%) +5ms (+ 9.55%) 57ms 74ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,999ms (± 0.45%) 1,990ms (± 0.79%) -9ms (- 0.45%) 1,963ms 2,024ms
Req 2 - geterr 746ms (± 0.72%) 745ms (± 0.63%) -1ms (- 0.13%) 737ms 757ms
Req 3 - references 71ms (± 2.01%) 75ms (± 1.85%) +3ms (+ 4.78%) 72ms 77ms
Req 4 - navto 219ms (± 0.50%) 216ms (± 0.58%) -3ms (- 1.42%) 213ms 219ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 270ms (± 0.63%) 283ms (± 1.56%) +14ms (+ 5.04%) 272ms 290ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 51682 10
Baseline main 10

Startup

Comparison Report - main..51682
Metric main 51682 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 117.55ms (± 0.36%) 119.68ms (± 0.55%) +2.13ms (+ 1.81%) 116.53ms 125.10ms
tsserver-startup - node (v16.17.1, x64)
Execution time 197.81ms (± 0.28%) 202.69ms (± 0.50%) +4.88ms (+ 2.47%) 196.44ms 212.66ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 192.24ms (± 0.31%) 196.72ms (± 0.38%) +4.48ms (+ 2.33%) 191.07ms 205.50ms
typescript-startup - node (v16.17.1, x64)
Execution time 177.21ms (± 0.30%) 181.51ms (± 0.36%) +4.30ms (+ 2.43%) 176.52ms 188.28ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-131-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 51682 10
Baseline main 10

Developer Information:

Download Benchmark

@@ -2803,18 +2738,20 @@ function createBinder(): (file: SourceFile, options: CompilerOptions) => void {
}
// falls through
case SyntaxKind.ThisKeyword:
// TODO: Why use `isExpression` here? both Identifier and ThisKeyword are expressions.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC this in a type position wasn't an expression, but was still parsed as a ThisKeyword for a period of time.

@Andarist
Copy link
Contributor

Andarist commented Dec 8, 2022

@Jack-Works I'm not sure how closely this relates to the stats posted by the bot here (most of them overlap but I'm not sure if they are 1 to 1) but you can call tsc --diagnostics and tsc --extendedDiagnostics

@rbuckton
Copy link
Member Author

rbuckton commented Dec 8, 2022

I'm curious. How I can run the perf on my own project?

The tool we use for benchmarking isn't public, but it essentially just runs tsc --extendedDiagnostics repeatedly on various projects and summarizes the results.

I'm planning to publish the tool I wrote to analyze V8 maps and deoptimizations in the near future, however.

@Jack-Works
Copy link
Contributor

System info:

Node 19.2.0, Windows 10.0.22623, AMD64 Family 25 Model 33 Stepping 0 AuthenticAMD ~3701 Mhz

Project scale:

Projects in scope:                         75
Projects built:                            71
Aggregate Files:                        64824
Aggregate Lines of Library:            716308
Aggregate Lines of Definitions:       8345732
Aggregate Lines of TypeScript:         216245
Aggregate Lines of JavaScript:              0
Aggregate Lines of JSON:                40878
Aggregate Lines of Other:                   0

Perf result:

5.0.0-dev.20221207 => npm:@typescript-deploys/pr-build@5.0.0-pr-51682-13

Aggregate Identifiers:                9923117
Aggregate Symbols:                    8689294 => 8689301
Aggregate Types:                       998455 => 998526
Aggregate Instantiations:             6990544
Aggregate Memory used:               2633329K => 2685920K
Aggregate Assignability cache size:    428849
Aggregate Identity cache size:          53133
Aggregate Subtype cache size:           49022
Aggregate Strict subtype cache size:    37922
Aggregate I/O Read time:                0.72s
Aggregate Parse time:                   2.23s => 2.32s
Aggregate ResolveModule time:           2.79s => 2.78s
Aggregate ResolveTypeReference time:    0.10s => 0.11s
Aggregate Program time:                10.15s => 10.23s
Aggregate Bind time:                    1.08s => 10.7s
Aggregate Check time:                  35.31s => 35.43s
Aggregate transformTime time:           5.92s => 5.81s
Aggregate Source Map time:              0.83s => 0.81s
Aggregate commentTime time:             1.19s => 0.16s
Aggregate printTime time:              19.76s => 19.60s
Aggregate Emit time:                   19.82s => 19.68s
Aggregate I/O Write time:               2.81s => 2.90s
Config file parsing time:               0.32s
Up-to-date check time:                  0.01s
Build time:                            90.30s => 90.60s

@rbuckton rbuckton marked this pull request as ready for review December 12, 2022 20:12
@@ -28,7 +28,7 @@ import {
BundleFileTextLikeKind,
CallExpression,
CallSignatureDeclaration,
CaseBlock,
canHaveLocals, CaseBlock,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no lint rule for this, but these should be columns.

Hopefully one day we can get formatting automated...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One day...
futuristic city

🤣 (sorry, I couldn't resist)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no lint rule for this, but these should be columns.

Hopefully one day we can get formatting automated...

I've also noticed that Organize Imports seems to be sorting __String to the end rather than the start.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's: #51733

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants